Standardization of network management across cloud computing environments and data control policies

ABSTRACT

Network management of cloud computing environments subject to different data control policies is standardized in a manner that ensures compliance with the data control policies. Executions services and source of truth services are located in a remote cloud computing environment separate from the cloud computing environments being managed. The execution services implement workflows to manage different aspects of the cloud computing environments, including monitoring, incident management, deployment, and buildout. The source of truth services provide network configuration information for the cloud computing environments to allow automated operation of the execution services. The execution services issue requests for management operations to device access services in the cloud computing environments. In response to the requests, the device access services obtain access control data to access the network devices and perform the management operations.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related by subject matter to the followingapplications: U.S. application Ser. No. 14/933,803, entitled INCIDENTMANAGEMENT TO MAINTAIN CONTROL OF RESTRICTED DATA IN CLOUD COMPUTINGENVIRONMENTS; U.S. application Ser. No. 14/933,815, entitled MAINTAININGCONTROL OVER RESTRICTED DATA DURING DEPLOYMENT TO CLOUD COMPUTINGENVIRONMENTS; U.S. application Ser. No. ______ (not yet assigned)(Attorney Docket Number 402093-US-NP/MFCP.274071), filed on even dataherewith and entitled MONITORING CLOUD COMPUTING ENVIRONMENTS WITH DATACONTROL POLICIES; U.S. application Ser. No. ______ (not yet assigned)(Attorney Docket Number 402094-US-NP/MFCP.274070), filed on even dataherewith and entitled SOFTWARE DEPLOYMENT TO NETWORK DEVICES IN CLOUDCOMPUTING ENVIRONMENTS WITH DATA CONTROL POLICIES; and U.S. applicationSer. No. ______ (not yet assigned) (Attorney Docket Number402090-US-NP/MFCP.274090), filed on even data herewith and entitledNETWORK BUILDOUT FOR CLOUD COMPUTING ENVIRONMENTS WITH DATA CONTROLPOLICIES. The aforementioned applications are assigned or underobligation of assignment to the same entity as this application, and areherein incorporated by reference in their entirety.

BACKGROUND

Cloud computing environments, including data centers, server farms andthe like, have become increasingly common to provide vast amounts ofcomputational and storage resources. For example, cloud computingenvironments have been utilized to store and retrieve vast amounts ofdata for various service applications (e.g., web applications, emailservices, search engine services, etc.). These networked systemstypically include a large number of nodes distributed throughout one ormore data centers, in which each node provides a physical machine or avirtual machine running on a physical host.

Due partly to the complexity and large number of the nodes that may beincluded within such cloud computing environments, resolving incidentsand deploying software updates can be a time-consuming and costlyprocess. Data control policies imposed on cloud computing environmentsalso contribute to the challenges of monitoring, incident management,and deployment. In particular, many cloud computing environments aresubject to data control policies that limit access to certain data andto the control plane, which allows for implementing changes to theproduction environment (i.e., the physical and logical environment wherecloud service infrastructure components providing services to customersare hosted). These data control policies may be driven by a variety offactors, such as, for instance, customer-driven requirements, laws, orindustry best practices. Such data control policies may restrict a givencloud computing environment to certain service-providing entities orpersonnel authorized to access certain data or the productionenvironment, geographical boundaries, or certain logical or physicalcomponents within a given production environment. The data controlpolicies dictate that certain restricted data must reside within aparticular cloud computing environment and not cross into otherconnected cloud computing environments or otherwise leave thatparticular cloud computing environment. By way of example to illustrate,customers in highly regulated industries such as healthcare may requirerestriction of their computing environment to certain screenedpersonnel. As another example, some customers may be subject toregulations that restrict the geographical boundaries in which cloudservices are provided or where restricted data is stored, processed, orboth. Such regulations may include the personnel authorized to haveaccess to restricted data and to the control plane of the productionenvironment. Complying with these data control policies poses challengesin how the cloud services are deployed and managed to maintain thecontrol over the restricted data.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

Aspects of the technology described herein generally relate tostandardizing the management of diverse cloud computing environmentsthat are subject to different data control policies. The cloud computingenvironments are managed remotely by execution services located in aremote cloud computing environment in a manner that ensures compliancewith data control policies by maintaining control of restricted data inthe cloud computing environments. The execution services implementworkflows that manage different aspects of the cloud computingenvironments, including monitoring, incident management, deployment, andbuildout. Source of truth services in the remote cloud computingenvironment provide network configuration information for the cloudcomputing environments to allow automated operation of the executionservices. To comply with data control policies, the execution servicesdo not have access to restricted data in the cloud computingenvironments, including access control data, such that the executionservices cannot directly interact with network devices. To performmanagement operations for network devices, the execution services issuerequests to device access services in each cloud computing environment.In response to the requests, the device access services obtain accesscontrol data to access the network devices and perform managementoperations. In this manner, restricted data is maintained within eachcloud computing environment, but management workflows can bestandardized across the various cloud computing environments.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects of the disclosure are described in detail below with referenceto the attached drawing figures, wherein:

FIG. 1 is a block diagram showing a system for managing cloud computingenvironments subject to data control polices from a remote cloudcomputing environment in accordance with aspects of the presentdisclosure;

FIG. 2 is a flow diagram showing a method for using an SNMP proxy toobtain telemetry data for a network device in a cloud computingenvironment in response to a request from an execution service in aremote cloud computing environment in accordance with aspects of thepresent disclosure;

FIG. 3 is a flow diagram showing a method for using a hardware proxy toperform an action on a network device in a cloud computing environmentin response to a request from an execution service in a remote cloudcomputing environment in accordance with aspects of the presentdisclosure;

FIG. 4 is a flow diagram showing a method for collecting telemetry datafrom network devices in a cloud computing environment at a monitoringservice in a remote cloud computing environment in accordance withaspects of the present disclosure;

FIG. 5 is a flow diagram showing a method for automatically deployingsoftware to one or more network devices in a cloud computing environmentfrom a remote cloud computing environment in accordance with aspects ofthe present disclosure;

FIG. 6 is a flow diagram showing a method for performing buildout in acloud computing environment from a remote cloud computing environment inaccordance with aspects of the present disclosure;

FIG. 7 is a flow diagram showing a method for configuring a networkdevice in accordance with aspects of the present disclosure;

FIG. 8 is a flow diagram showing a method for validating configurationof a network device in accordance with aspects of the presentdisclosure; and

FIG. 9 is a block diagram of an exemplary computing environment suitablefor use in implementing aspects of the present disclosure.

DETAILED DESCRIPTION

The subject matter of the present disclosure is described withspecificity herein to meet statutory requirements. However, thedescription itself is not intended to limit the scope of this patent.Rather, the inventors have contemplated that the claimed subject mattermight also be embodied in other ways, to include different steps orcombinations of steps similar to the ones described in this document, inconjunction with other present or future technologies. Moreover,although the terms “step” and/or “block” may be used herein to connotedifferent elements of methods employed, the terms should not beinterpreted as implying any particular order among or between varioussteps herein disclosed unless and except when the order of individualsteps is explicitly described.

As noted above, data control policies on cloud computing environmentsoften limit access to certain data and to the control plane to implementchanges to the production environment (i.e., the physical and logicalenvironment where cloud service infrastructure components providingservices to customers are hosted). In accordance with some data controlpolicies, data stored by a cloud computing environment includes bothnon-restricted data and restricted data. While access to non-restricteddata may be more generally available, restricted data is maintainedwithin the cloud computing environment and access to restricted data isavailable only to individuals who satisfy the requirements dictated bythe data control policies. As used herein, the term “operatingpersonnel” is used to refer to the individuals who have persistentaccess to, and do not require pre-approval to access, restricted data.The individuals who are considered operating personnel may varydepending on the applicable data control policies of the cloud computingenvironment. By way of example only, operating personnel may be requiredto reside in the country at which the cloud computing environment islocated and have passed screening requirements (e.g.,background/security clearance checks). Operating personnel may be athird party entity, authorized personnel either within a given entity oracross multiple entities. Operating personnel is typically defined bythe cloud service provider, but in some instances, operating personnelmay be defined by the customer.

In contrast to operating personnel, “DevOps personnel” includeindividuals from engineering teams of a cloud service provider(including subsidiaries, affiliates, vendors, etc.) who do not haveaccess to “restricted data” and unlimited access to the control plane ofa cloud computing environment. In some instances, the DevOps personnelmay not reside within the country within which the cloud computingenvironment is located and may not be subject to the same securityscreening requirements applied to the operating personnel.

As used herein, “restricted data” includes any data that must bemaintained within a cloud computing environment and/or whose access isrestricted to and/or controlled by operating personnel as dictated bydata control policies applicable to that cloud computing environment. Byway of example only and not limitation, restricted data may includecustomer content/data, end user identifiable information, and accesscontrol data. “Customer content” is defined as content directly createdby customer users and all data, including all text, sound, software orimage files that customers provide, or are provided on customers'behalf, through use of the services. This includes but is not limitedto: email body (full or partial), email attachment body, information inthe body of a file, IM or voice conversations, customer generated blobor structured storage data, customer's binaries running in virtualmachines, customer-owned security information/secrets (certificates,encryption keys, storage keys, customer address list data (name, emailaddress(es), office address, phone numbers, manager/direct reports, jobtitle, distribution group memberships), network packet payloads,database contents, service bus message contents, etc. “End useridentifiable information” is defined as data unique to a user, orgenerated from their use of the service; is linkable to an individualuser and does not include customer content. This includes but is notlimited to: user specific Internet Protocol (IP) address, email address,email subject line or email attachment name, user name, display name,office number, employee ID, address book data, behavioral/usage datathat is linkable to an individual user, location information, machinename, etc. “Access control data” is used to manage access to networkdevices and/or other types of data or functions within the cloudcomputing environment, including access to customer content or end useridentifier information. This includes passwords, security certificates,and other authentication-related data, such as: passwords to platformcomponents; private keys of certificates used to manage platformcomponents; and SNMP community strings.

Alternatively, “non-restricted” data may be more generally accessibleoutside of the cloud computing environment and not limited to access byoperating personnel. By way of example only and not limitation,non-restricted data may include account/administrator data, paymentdata, organization identifiable information, and system metadata.“Account/administrator data” is information about administratorsprovided during sign-up, purchase, or administration of the services,such as: name of the customer company name (e.g. “Contoso”), InternetDomain Name of the customer (without user name; e.g. “contoso.cn”),customer company billing address, name, user name, email address ofadministrator of a service hosting a service, IP address of such anadministrator's computer or of customer servers (i.e., not tied to enduser), etc. “Payment Data” is information about payment instruments suchas credit card details. It is subject to other security precautions butmay not considered “restricted” for access restrictions addressedherein. “Organization identifiable information” is defined as data thatcan be used to identify a particular tenant (generally configuration orusage data), is not linkable to an individual user, and does not containcustomer content. This may include: tenant ID, customer subscriptionIDs, aggregated behavioral/usage data associable with a tenant but not auser, tenant usage data, tenant IP addresses (e.g. IP Addressesassociated with customer's virtual machines or on premise servers (butnot individual end users), etc. “System metadata” comprisesconfiguration and operations data, such as: service logs (provided theydon't contain restricted data), technical information about asubscription (e.g. service topology), technical information about atenant (e.g. customer role name), configuration settings/files, servicestatus, performance metrics, telemetry data, IP addresses used forinternet transit service (firewall, netflow, sflow), etc.

The data control policies limiting access to restricted data and theability to make certain changes to the production environment of cloudcomputing environments poses challenges to cloud service providers. Inparticular, management of a cloud service requires monitoring networkdevices and managing incidents, which may include, for instance,maintenance tasks, deployment incidents, live site incidents, customerreported incidents, and support requests. Additionally, management of acloud service requires periodic updates and patches to be deployed tothe production environment. When increased capacity is needed,additional network devices need to added to the production environmentand properly configured through network buildout. In the context of acloud computing environment in which access to restricted data and thecontrol plane are maintained within the cloud computing environmentand/or limited to operating personnel based on data control policies, itmay be difficult to properly provide monitoring, incident management,software/firmware deployment, and network buildout as the number andavailable expertise of the operating personnel may not be sufficient toproperly maintain the cloud computing environment. Additionally,different cloud computing environments provided by a cloud serviceprovider can have different configurations and be subject to differentdata control policies, further complicating network management as eachcloud computing environment can have its own management services.

Aspects of the technology described herein are directed to technologicalimprovements that provide for the standardization of management of cloudcomputing environments while complying with different data controlpolicies applicable to those cloud computing environments. In accordancewith some aspects of the present disclosure, execution services areprovided in a remote cloud computing environment separate from the cloudcomputing environments being managed. The execution services providemanagement workflows that are standardized across the various cloudcomputing environments, such as workflows to perform monitoring,incident management, software deployment, and network buildout.

To comply with data control policies, the execution services do not havethe ability to obtain access control data or other restricted data thatis maintained in each of the cloud computing environments. As such, theexecution services can be instantiated once outside of the confines ofeach cloud computing environment. However, because the executionservices cannot obtain access control data, the execution servicescannot directly access network devices in the cloud computingenvironments. Instead, device access services are provided within eachcloud computing environment. Because each device access service islocated within the confines of a cloud computing environment, eachdevice access service can obtain access control data to access networkdevices and perform management operations. The execution services in theremote cloud computing environment send requests for data and otheroperations to the device access services. In response to the requests,the device access services obtain access control data in order to accessnetwork devices in the cloud computing environment and obtain therequested data and perform the requested operations. The device accessservices can return non-restricted data back to the execution services.

The workflows of some execution services can be automated based onnetwork configuration information for the cloud computing environments.Because network configuration information does not include restricteddata, the network configuration information can be collected andmaintained by source of truth services in the remote cloud computingenvironment. The source of truth services make the network configurationinformation available to the execution services, which can determineactions that need to be taken on network devices in the cloud computingenvironments based on the configuration information.

Accordingly, aspects of the present disclosure provide forstandardization of management of a cloud service provider's cloudcomputing environments while maintaining compliance with different datacontrol policies. Management workflows provided by execution servicesare decoupled from access control data and network device interaction,allowing the execution services to be instantiated remotely from thecloud computing environments, which also minimizes the network DevOpsfootprint in each of the cloud computing environments. This allows forthe same management capabilities to be applied across the differentcloud computing environments in a compliant and scalable way. Managementof the cloud computing environments can be controlled remotely whilepreventing unapproved access to restricted data and/or to the controlplane to implement changes to the production environment of each cloudcomputing environment.

With reference to FIG. 1, a block diagram is provided illustrating anexemplary system 100 in which some aspects of the present disclosure maybe employed. It should be understood that this and other arrangementsdescribed herein are set forth only as examples. Other arrangements andelements (e.g., machines, interfaces, functions, orders, and groupingsof functions, etc.) can be used in addition to or instead of thoseshown, and some elements may be omitted altogether. Further, many of theelements described herein are functional entities that may beimplemented as discrete or distributed components or in conjunction withother components, and in any suitable combination and location. Variousfunctions described herein as being performed by one or more entitiesmay be carried out by hardware, firmware, and/or software. For instance,various functions may be carried out by a processor executinginstructions stored in memory.

Among other components not shown, the system 100 includes cloudcomputing environments 102A, 102B, operator devices 104A, 104B, a remotecloud computing environment 106, and a DevOps device 108. It should beunderstood that the system 100 shown in FIG. 1 is an example of onesuitable computing system architecture. Each of the components shown inFIG. 1 may be implemented via any type of computing device, such ascomputing device 900 described with reference to FIG. 9, for example.The components may communicate with each other via a network, which mayinclude, without limitation, one or more local area networks (LANs)and/or wide area networks (WANs). Such networking environments arecommonplace in offices, enterprise-wide computer networks, intranets,and the Internet. It should be understood that any number of cloudcomputing environments, operator devices, remote cloud computingenvironments, and DevOps devices may be employed within the system 100within the scope of the technology described herein. Each may comprise asingle device or multiple devices cooperating in a distributedenvironment. Additionally, other components not shown may also beincluded within the network environment.

Each of the cloud computing environments 102A, 102B includes aproduction environment 110A, 110B, which comprises the physical andlogical environment where cloud service infrastructure componentsproviding services to customers are hosted. This includes systems thatstore/process both restricted and non-restricted data. Each of theproduction environments 110A, 110B is made up of a number of resources112A, 112B. These resources 112A, 112B include physical network devices(e.g., servers, storage devices, memory, routers, etc.), as well assoftware running on and data stored on the physical devices. AlthoughFIG. 1 illustrates a system 100 that includes two separate cloudcomputing environments 102A, 102B, it should be understood that anynumber of cloud computing environments can be included.

The cloud computing environments 102A, 102B are subject to data controlpolicies that limit access to restricted data and to the control planesthat allow for implementing changes to the production environments 110A,110B. The applicable data control policies vary between the cloudcomputing environments 102A, 102B. The data control policies applicableto each cloud computing environment 102A, 102B can be dictated by anumber of different factors, such as for instance, customerrequirements, industry standards, and applicable laws based on thelocation at which each cloud computing environment 102A, 102B issituated.

The system 100 is configured in a manner such that the cloud computingenvironments 102A, 102B can be managed externally from the remote cloudcomputing environment 106 and/or DevOps devices, such as the DevOpsdevice 108, without violating the data control policies of the cloudcomputing environments 102A, 102B. Additionally, in accordance with thetechnology described herein, aspects of managing the different cloudcomputing environments 102A, 102B are standardized despite each of thecloud computing environments 102A, 102B being subject to different datacontrol policies. This allows for the same management capabilities to beapplied across the different cloud computing environments 102A, 102B.

The remote cloud computing environment 106 can be any public or privatecloud computing environment with a network of devices providingcomputing resources. To facilitate standardized management of the cloudcomputing environments 102A, 102B while complying with the applicabledata control policies, the remote cloud computing environment 106includes a number of execution services 114 and source of truth services116. The execution services 114 are a collection of services thatimplement workflows to manage different aspects of the cloud computingenvironments 102A, 102B. The execution services 114 do not havepersistent access to restricted data in the cloud computing environments102A, 102B, including access control data required to access networkdevices in the cloud computing environments 102A, 102B. Because theexecution services 114 do not have persistent access to restricted data,the execution services 114 don't need to be instantiated in each cloudcomputing environment 102A, 102B. Instead, the execution services 114can be instantiated once and centralized in the remote cloud computingenvironment 106 outside of the boundaries of each of the cloud computingenvironments 102A, 102B. A single instance of each execution service 114can be configured to communicate with and manage aspects of thedifferent cloud computing environments 102A, 102B. Although theexecution services 114 can be instantiated once (e.g., in the remotecloud computing environment 106), it should be understood that in someconfigurations, multiple instances of the execution services 114 canreside in various locations (e.g., other remote cloud computingenvironments not shown in FIG. 1).

The execution services 114 can provide a variety of different types ofservices to manage aspects of the cloud computing environments 102A,102B. As shown in FIG. 1, the execution services 114 can include, amongother things, a monitoring service 118, a deployment service 120, and abuildout service 122. Each of these services 118, 120, 122 will bedescribed in further detail below. It should be understood that each ofthese services 118, 120, 122 can comprise either a single service or acollection of services. Additionally, it should be understood that theexecution services 114 can include a variety of other services not shownin FIG. 1.

Some execution services 114 allow for automation in order to manage thenetwork devices at cloud scale. In other words, it may be infeasible tomanually perform some services, such as device monitoring and deploymentof software, when there is a large number of network devices in thecloud computing environments 102A, 102B. However, not all aspects may befully automated, and some execution services 114 can provide userinterfaces that a DevOps personnel can access, for instance, using theDevOps device 108.

To support automation of execution services 114 (e.g., automateddeployment and automated buildout), source of truth services 116 operateto collect network configuration information for the cloud computingenvironments 102A, 102B and serve as repositories of the information.Similar to the execution services 114, the source of truth services 116do not have access to restricted data from the cloud computingenvironments 102A, 102B. As such, the source of truth services 116 don'tneed to be instantiated in each cloud computing environment 102A, 102B.Instead, the source of truth services 116 can be instantiated once andcentralized in the remote cloud computing environment 106 outside of theboundaries of the cloud computing environments 102A, 102B. A singleinstance of each source of truth service 116 can collect networkconfiguration information for all of the different cloud computingenvironments 102A, 102B. Although the source of truth services 116 canbe instantiated once (e.g., in the remote cloud computing environment106), it should be understood that multiple instances of the source oftruth services 116 can reside in various locations (e.g., other remotecloud computing environments not shown in FIG. 1).

In accordance with some aspects of the present technology, the source oftruth services 116 can include, among other things, a network graphservice 124 and a network state service 126. The network graph service124 builds a network graph from network configuration information. Thenetwork graph describes the network devices and links connecting thenetwork devices in the cloud computing environments 102A, 102B. Thisprovides information such as link-level attributes, data-levelattributes, IP addresses assigned to network devices, and networkrouting (e.g., preferred and/or short routes).

The network state service 126 hosts information regarding the state ofnetwork devices in the cloud computing environments 102A, 102B. Eachnetwork device can be modeled in different states (e.g., an observedstate, proposed target state, and target state). The network stateservice 126 enables the decoupling of the logic of workflows of theexecution services 114 from the logic of interacting with networkdevices. The execution services 114 can be stateless in terms of networkdata and focus on proposing new network states based on observed states.The execution services 114 don't need to know how the observed statesare collected from heterogeneous network devices in the cloud computingenvironments 102A, 102B or how to update network devices towards targetstates. The execution services 114 can just change proposed targetstates.

While network configuration information can move outside of the cloudcomputing environments 102A, 102B, access control data required toaccess network devices in the cloud computing environments 102A, 102B ismaintained within the boundary of each of the cloud computingenvironments 102A, 102B to comply with data control policies. Forinstance, as shown in FIG. 1, the cloud computing environments 102A,102B includes access control data stores 128A, 128B for storing accesscontrol data specific to the cloud computing environments 102A, 102B.For instance, the access control data store 128A stores access controldata specific to the cloud computing environment 102A; and the accesscontrol data store 128B stores access control data specific to the cloudcomputing environment 102B. Because access control data is maintainedwithin each of the cloud computing environments 102A, 102B, the accesscontrol data is not available to the execution services 114. As such,the execution services 114 cannot directly access the network devices inthe cloud computing environments 102A, 102B, helping to ensurecompliance with the data control policies applicable to the cloudcomputing environments 102A, 102B. Instead, the execution services 114interact with device access services 130A, 130B located within the cloudcomputing environments 102A, 102B.

Each of the cloud computing environments 102A, 102B includes its ownrespective set of device access services 130A, 130B. The device accessservices 130A, 130B interact directly with network devices in theirrespective cloud computing environment 102A, 102B using access controldata for those operations. As such, the device access services 130A,130B run on devices within the their respective cloud computingenvironment 102A, 102B, keeping the access control data within theconfines of each cloud computing environment 102A, 102B.

On the one side, the device access services 130A, 130B receive requestsfrom execution services 114 to take actions on network devices in theirrespective cloud computing environments 102A, 102B. For instance, APIscan be used by the execution services 114 to submit requests forspecific actions to the device access services 130A, 130B. In responseto the requests, the device access services 130A, 130B obtain accesscontrol data from their respective access control data stores 128A, 128Band access the network devices using the access control data to performthe requested actions. The device access services 130A, 130B can returnresponses back to the execution services 114 that can include, forinstance, requested data or an indication that an action was performedon the network devices. Data returned to execution services 114 may bescrubbed before it is sent to remove any restricted data.

Each of the cloud computing environments 102A, 102B includes arespective access control service 132A, 132B that controls externalaccess to its respective cloud computing environment 102A, 102B. Inaccordance with some configurations, there is no trust relationshipbetween each of the cloud computing environments 102A, 102B and theremote cloud computing environment 106 at the directory level (i.e.,directory services don't trust each other). Instead, requests arebrokered through claims-based access control. As such, the cloudcomputing environments 102A, 102B don't need to trust the remote cloudcomputing environment 106 (or vice versa). Instead, each of the cloudcomputing environments 102A, 102B are configured to trust the claimssubmitted by the remote cloud computing environment 106 (and viceversa). For any request, a claim is submitted. The access controlservice 132A, 132B of the cloud computing environment 102A, 102Breceiving the request evaluates the claim to determine whether to acceptthe claim.

The claims-based approach also helps to isolate the cloud computingenvironments 102A, 102B from one another by preventing trust between thecloud computing environments 102A, 102B. Because there is notransitivity of trust between one of the cloud computing environments102A, 102B to another of the cloud computing environments 102A, 102Bthrough the remote cloud computing environment 106, there is no way totraverse from one of the cloud computing environments 102A, 102B toanother of the cloud computing environments 102A, 102B through theremote cloud computing environment 106. Each of the cloud computingenvironments 102A, 102B is configured to trust claims of the remotecloud computing environment 106 but does not trust claims of the othercloud computing environments 102A, 102B.

As shown in FIG. 1, the device access services 130A, 130B can include,among other things, a respective SNMP proxy 134A, 134B, hardware proxy136A, 136B, and UDP proxy 138A, 138B. Each SNMP proxy 134A, 134Boperates to provide telemetry data for network devices of the cloudcomputing environment 102A, 102B in which it resides. Each SNMP proxy134A, 134B does not perform any configuration on network devices butonly provides telemetry data. The telemetry data can include any linkutilization information and other system metadata regarding how networkdevices are functioning. FIG. 2 provides a flow diagram illustrating amethod 200 for using the SNMP proxy 134A to obtain telemetry data for anetwork device in the cloud computing environment 102A in response to arequest from an execution service 114. Each block of the method 200 andother methods described herein comprises a computing process that may beperformed using any combination of hardware, firmware, and/or software.For instance, various functions may be carried out by a processorexecuting instructions stored in memory. The methods may also beembodied as computer-usable instructions stored on computer storagemedia. While the method 200 is described in context of the cloudcomputing environment 102A, it should be understood that a similarprocess can be used with the cloud computing environment 102B.

As shown at block 202, the SNMP proxy 134A receives a request fortelemetry data from an execution service 114 in the remote cloudcomputing environment 106. The request can be made, for instance, viaAPIs available to the execution services 114. As noted above, when aclaims-based approach is employed, the request from the executionservice 114 includes a claim, which is evaluated by the access controlservice 128A of the cloud computing environment 102A receiving therequest to determine whether to accept the claim. The request canspecify information, such as the type of telemetry data requested andthe network device for which the telemetry data is requested. Forexample, the request could be a command to get interface counters andspecify a network device name (e.g., using an SNMP object identifier(OID)).

In response to the request, the SNMP proxy 134A obtains a SNMP communitystring for the network device from the access control data store 128A,as shown at block 204. As is known in the art, SNMP community stringsare used to gain access to telemetry data from network devices. The SNMPproxy 134A then issues a request to the network device, as shown atblock 206. The request can comprise an SNMP get request that specifiesthe particular telemetry data requested (e.g., get interface counters)and includes the SNMP community string.

The SNMP proxy 134A receives a response from the network device thatincludes the requested telemetry data, as shown at block 208. In someinstances, the SNMP proxy 134A decodes the response, for instance usinga management information base (MIB), to obtain the telemetry data, asshown at block 210. The SNMP proxy 134A then sends the telemetry data tothe execution service 114 as a response to the original request from theexecution service, as shown at block 212.

While each of the SNMP proxies 134A, 134B operates to obtain telemetrydata from devices, each of the hardware proxies 134A, 134B is configuredto generally perform actions on network devices by issuing commands tothe network devices, for instance, by logging into the network devicesusing access control data for each network device. The actions caninclude obtaining telemetry data. For instance, in some instances, anSNMP proxy 134A, 134B cannot be used to obtain telemetry data (e.g., ifthe SNMP OIDs are too complex to pass out or a network device doesn'timplement an SNMP OID). In such instances, a hardware proxy 136A, 136Bcan be used to obtain telemetry data from a network device. In additionto obtaining telemetry data from network devices, the hardware proxies136A, 136B can be used to issue commands to configure network devices.

Turning to FIG. 3, a flow diagram is provided that illustrates a method300 for using the hardware proxy 136A to perform an action on a networkdevice in response to a request from an execution service 114. While themethod 300 is described in context of the cloud computing environment102A, it should be understood that a similar process can be used withthe cloud computing environment 102B.

As shown at block 302, the hardware proxy 136A receives a request toperform an action on a network device from an execution service 114 inthe remote cloud computing environment 106. The request can be made, forinstance, via APIs available to the execution services 114. As notedabove, when a claims-based approach is employed, the request from theexecution service 114 includes a claim, which is evaluated by the accesscontrol service 128A of the cloud computing environment 102A receivingthe request to determine whether to accept the claim. The request canspecify information, such as the type of action requested and thenetwork device for which the action is requested.

In response to the request, the hardware proxy 136A obtains accesscontrol data necessary to access the network device from the accesscontrol data store 128A, as shown at block 304. The access control datadepends on the type of device being accessed. The hardware proxy 136Auses the access control data to log into the network device, as shown atblock 306. Once logged into the network device, the hardware proxy 136Aperforms the requested action on the network device, as shown at block308. The action may be performed in different manners depending on thetype of network device. The hardware proxy 136A is configured tointerface with the different types of network devices within the cloudcomputing environment 102A in which it is located. For instance, thehardware proxy 136A can issue commands to the network device via acommand-line tool such as a device console.

The hardware proxy 136A obtains data regarding the requested action onthe network device, as shown at block 310. For example, in instances inwhich the requested action is to obtain telemetry data for the networkdevice, the hardware proxy 136A receives the requested telemetry data.In instances, in which the requested action is to perform configurationaction on the network device, the hardware proxy 136A obtains dataregarding whether the configuration action has been successfullycompleted. In some configurations, the hardware proxy 136A obtains thedata regarding the requested action using a screen scraping capabilityto read output from the network device via a command-line tool used tointerface with the network device.

In some instances, the data obtained regarding the requested actionincludes restricted data. As such, the hardware proxy 136A evaluates thedata for restricted data, as shown at block 312. If it is determined atblock 314 that the data includes any restricted data, the restricteddata is removed from the data, as shown at block 316. For instance, therestricted data can be encrypted or replaced with placeholders. As shownat block 318, the hardware proxy 136A sends the data to the executionservice 114 as a response to the original request from the executionservice, as shown at block 318.

Network devices in each of the cloud computing environments 102A, 102Bpush additional telemetry data to a UDP proxy 138A, 138B. The telemetrydata pushed from network devices to each UDP proxy 138A, 138B caninclude, for instance, IPFIX data, SNMP trap data, AAA data, and syslogsdata. Telemetry data pushed to the UDP proxy 138A, 138B that does notcontain restricted data can be sent to execution services 114 in theremote cloud computing environment 106. For instance, SNMP trap data,AAA data, and syslog data do not contain restricted data and thereforecan be passed out of each of the cloud computing environments 138A, 138Bto an execution service 114, such as the monitoring service 118.Telemetry data can also be sent from each UDP proxy 138A, 138B to arespective UDP collector 140A, 140B in each of the cloud computingenvironments 102A, 102B. Since each UDP collector 140A, 140B is locatedin a respective cloud computing environment 102A, 102B, each of the UDPcollectors 140A, 140B can receive all telemetry data, includingrestricted data, from a respective UDP proxy 138A, 138B. For instance,IPFIX data can contain potential end user IP address information, whichcan be considered as restricted data. Each of the UDP collectors 140A,140B can serve as a repository of telemetry data from its respective UDPproxy 138A, 138B. Additionally, each of the UDP collectors 140A, 140Bcan remove restricted data from the telemetry data (e.g., by encryptingthe restricted data or replacing the restricted data with placeholders)and pass the “cleansed” telemetry data outside of its respective cloudcomputing environment 102A, 102B, for instance, to execution services114 in the remote cloud computing environment 106.

Monitoring Network Devices

As noted above, one of the execution services 114 in the remote cloudcomputing environment 106 is a monitoring service 118, which providesthe capability to monitor network devices in the cloud computingenvironments 102A, 102B from the remote cloud computing environment 106.The monitoring service 118 performs automated operations to collect datafrom the cloud computing environments 102A, 102B. The monitoring service118 also provides user interfaces (e.g., charts, etc.) that allow forthe collected data (e.g., utilization, telemetry data, etc.) to bepresented to DevOps personnel (e.g., on the DevOps device 108). TheDevOps personnel can employ the user interfaces to view the data atdifferent levels of aggregations including drilling down to view data oneach device/node from each of the cloud computing environments 102A,102B. Data collected by the monitoring service 118 can also be used toautomatically identify incidents. For instance, thresholds can bedefined for different sets of monitoring tools. When data is identifiedthat meet a threshold, an incident is triggered so appropriatemitigations can be implemented. In some instances, the incidentmanagement can be performed automatically by an execution service 114initiating a workflow to resolve the incident. In other instances, theincident is reported to DevOps personnel who manually resolve theincident.

Turning to FIG. 4, a flow diagrams is provided that illustrates a method400 to collect data from network devices in the cloud computingenvironment 102A at the monitoring service 118 in the remote cloudcomputing environment 102. While the method 400 is described in contextof the cloud computing environment 102A, it should be understood that asimilar process can be used with the cloud computing environment 102B.

As shown at block 402, the monitoring service 118 sends requests fordata to the SNMP proxy 134A and the hardware proxy 136A. As noted abovewith reference to FIGS. 2 and 3, the requests can be made via APIs madeavailable to the monitoring service 118 and can be made using aclaims-based approach in which the requests include claims that areevaluated by the access control service. Each request can identifyrequested data and network device(s) for which the data is requested.

As shown at block 404, the SNMP proxy 134A obtains data from networkdevices in response to the requests the SNMP proxy 134A receives fromthe monitoring service 118. The SNMP proxy 134A can obtain the datausing the approach described above with reference to FIG. 2.Additionally, as shown at block 406, the hardware proxy 136A obtainsdata from network devices in response to the requests the hardware proxy136A receives from the monitoring service 118. The hardware proxy 136Acan obtain the data using the approach described above with reference toFIG. 3.

The monitoring service 118 receives data from the SNMP proxy 134A andthe hardware proxy 136A in response to the requests, as shown at block408. In some configurations, the monitoring service 118 receivesadditional data from the UDP proxy 138A and/or UDP collector 140A, asshown at block 410. The monitoring service 118 provides user interfacesto present the received data to DevOps personnel, as shown at block 412.The user interfaces can allow the DevOps personnel to view the data atthe individual network device level as well as various other levels ofaggregation.

Automated Software Deployment to Network Devices

One area presenting a potential issue for complying with the datacontrol policies of cloud computing environments is the deployment ofsoftware to network devices. This could include, for instance, thedeployment of software updates, firmware updates, patches, bug fixes, orother software deployments to maintain and/or update network devices inthe cloud computing environments. In particular, if DevOps personneland/or execution services external to the cloud computing environmentare tasked with the deployment of software to the network devices in thecloud computing environment, it is important to ensure that the softwaredeployments do not make changes that would allow DevOps personnelpersistent access to restricted data or otherwise allow access torestricted data outside of the cloud computing environment. Accordingly,some embodiments are directed to techniques that facilitate theautomatic deployment of software to network devices in the cloudcomputing environments 102A, 102B by a deployment service 120 in theremote cloud computing environment 106 in a manner that maintainsrestricted data within the cloud computing environments 102A, 102B. Aswill be discussed in further detail below, the software is approved byoperating personnel before deployment. If approved by operatingpersonnel, a release is then automatically deployed to network devicesin one or both of the cloud computing environments 102A, 102B. As such,the operating personnel is not required to deploy the release but maycontrol the deployment.

With reference now to FIG. 5, a flow diagram is provided illustrating amethod 500 for automatically deploying software to one or more networkdevices in the cloud computing environment 102A using the deploymentservice 120 in the remote cloud computing environment 106. While themethod 500 is described in context of the cloud computing environment102A, it should be understood that a similar process can be used withthe cloud computing environment 102B.

As shown at block 502, the deployment service 120 determines thatsoftware needs to be deployed on a network device in the cloud computingenvironment 102A. For instance, the deployment service 120 can consultsource of truth data available from the source of truth services 116 todetermine that software on the network device needs to be updated. Thismay be based on state information available for network devices. Forexample, the state information may reflect that the observed state for anetwork device differs from an expected state. In some instances, thedeployment service 120 may determine that software should be deployed toa number of network devices. For instance, the deployment service 120may determine that all network devices running an earlier firmwareversion should be updated to the current firmware version. As such, allnetwork devices running an earlier firmware version can be determinedbased on source of truth data and those network devices identified forupdate to the current firmware version.

Based on determining software needs to be deployed on the networkdevice(s), the deployment service 120 sends a deployment request forapproval by an operator via an approval service 142A (similar approvalservice 142B in the other cloud computing environment 102B), as shown atblock 504. Although the approval service 142A is shown as part of thecloud computing environment 102A in FIG. 1, it should be understood thatthe approval service 142A can be located externally. The deploymentrequest is provided to the operating personnel on the operator device104A and includes information regarding the software to be deployed andthe network device(s) to which the software is to be deployed. Theinformation included in the deployment request may include the softwareitself, a link to access the software, and/or specifications thatdescribe the software with sufficient detail for the operating personnelto understand the software and ensure that the software will not permitaccess to restricted data.

The operating personnel reviews the deployment request and selectswhether to approve or disapprove the software deployment. This gives theoperating personnel the opportunity to ensure that the softwaredeployment will not make changes to the cloud computing environment thatwould provide external access to restricted data. In instances in whichthe same software deployment is requested for multiple devices, theprocess can include a batch approval in which the operator can decidewhether to approve the software deployment for all the identifiednetwork devices collectively (as opposed to having to approve thesoftware deployment for each network device individually). For instance,the operating personnel can provide a batch approval for an upgrade to acurrent firmware version for all network devices with an earlier versionof the firmware.

A determination is made at block 506 regarding whether the operatingpersonnel approves the software deployment. If the software deploymentis disapproved, a notification of the disapproval is provided to DevOpspersonnel (e.g., via the DevOps device 108), as shown at block 508.Alternatively, if it is determined that the software deployment isapproved, the deployment service 120 issues a request to the hardwareproxy 136A to update the network device, as shown at block 510. In someinstances, the deployment service 120 can schedule the softwaredeployment to one or more network devices in order to maintain networkconnectivity. The deployment service 120 can consult networkconnectivity data available from the source of truth data to determinewhich devices can be taken offline at a time. For example, if thedeployment service determines that updates are being made to networkdevices to one critical part of the cloud computing environment 102A,the deployment service 120 can determine that the software deployment tothe network devices is to occur serially to avoid network outage.

In response to the request, the hardware proxy 136A obtains accesscontrol data from the access control data store 128A in order to accessthe network device, as shown at block 512. Using the access controldata, the hardware proxy 136A logs onto the network device, as shown atblock 514, and issues commands to deploy the software to the networkdevice, as shown at block 516. Different network devices may havedifferent methods for performing software deployments. Accordingly, thehardware proxy 136A can determine the type of network device on whichthe software deployment is being made and issue the appropriate commandsto perform the software deployment for that type of network device. Theprocess can include a file transfer for the software to the networkdevice. The software could be located where it can be communicated tothe cloud computing environment 102A. This could include a secure fileshare on the remote cloud computing environment 106 or another location.Once the software has been successfully deployed on a network device,source of truth data for the network device can be updated to reflectthe current state of the network device.

Network Buildout

Adding new network capacity to a cloud computing environment (referredto herein as “network buildout”) is typically an on-going effort.Network buildout includes adding new network devices to the cloudcomputing environment that involves both human-performed steps, such ascabling operations, and automated steps to configure the networkdevices. In accordance with some aspects of the present technology, abuildout service 122 in the remote cloud computing environment 106performs buildout in the cloud computing environments 102A, 102B byproviding an automated workflow that integrates the human performedoperations with the automated operations to configure the networkdevices. The buildout service 122 can track the lifecycle of the networkbuildout on a device-by-device basis, controlling operations andvalidating the proper configuration of the network devices.Additionally, the buildout service 122 can provide user interfaces toallow DevOps personnel (e.g., via the DevOps device 108) to view theprogress of a buildout.

Because the buildout service 122 is located outside of the cloudcomputing environments 102A, 102B, configuration and validationoperations are performed through the hardware proxy 136A and the SNMPproxy 138A, as will be described in further detail below. This ensurescompliance with data control policies by maintaining restricted datawithin the confines of each of the cloud computing environments 102A,102B.

FIG. 6 provides a flow diagram illustrating a method 600 forautomatically performing buildout in the cloud computing environment102A using the buildout service 122 in the remote cloud computingenvironment 106. While the method 600 is described in context of thecloud computing environment 102A, it should be understood that a similarprocess can be used with the cloud computing environment 102B.

As shown at block 602, the buildout service 122 identifies an occurrenceof network buildout in the cloud computing environment 102A. Forinstance, new network devices could be added to the network graphavailable from the network graph service 124 and marked with a specialattribute indicating that the network devices are being added as part ofa network buildout. The buildout service 122 could identify the networkdevices for buildout based on this indication.

Once the buildout is identified, the buildout service 122 starts aworkflow to initiate the buildout work, as shown at block 604. As notedabove, the buildout work includes both manual operations performed byoperating personnel at the cloud computing environment 102A (e.g.,cabling operations) and automated operations. The buildout service 122manages the manual operations, as shown at block 606. For instance, thebuildout service 122 can use a ticketing system in which actions itemsare sent to operating personnel as tickets to prompt the operatingpersonnel to perform those actions in the cloud computing environment102A. This could include creating tickets, updating tickets, and closingtickets as actions items are completed by operating personnel.

To facilitate the automated configuration of network devices for thebuildout, the buildout service 122 obtains configuration templates forthe network devices, as shown at block 608. The configuration templatesare generated within the remote cloud computing environment 106. Forinstance, the configuration templates can be generated by the source oftruth services 116 based on data defining the network architecture. Theconfiguration templates can be based on network device type and areintended to be populated with specific values in order to configure thenetwork devices.

In some instances, the buildout service 122 populates the configurationtemplates with values from non-restricted data available outside of thecloud computing environment 102A, as shown at block 610. The buildoutservice 122 sends configuration requests to the hardware proxy 136A inorder to configure the network devices involved in the buildout, asshown at block 612. This could include a separate request for eachnetwork device to be configured and/or batch requests to configuremultiple network devices. Each configuration request includes aconfiguration template for use in configuring one or more networkdevices.

In response to the configuration requests, the hardware proxy 136Aconfigures the network devices using the configuration templates, asshown at block 614. A method 700 for configuring a network device isshown in FIG. 7. To configure a given network device, the hardware proxy136A updates the configuration template for the network device withvalues from restricted data maintained in the cloud computingenvironment 102A, as shown at block 702. Additionally, the hardwareproxy 136A obtains access control data for the network device and usesthe access control data to log into the network device, as shown atblock 704. The hardware proxy 136A issues commands to the network deviceto configure the network device based on the populated configurationtemplate, as shown at block 706.

Returning to FIG. 6, after configuring network devices, the buildoutservice 122 also validates the configuration of the network devices, asshown at block 616. This could include various different validations,such as, for instance, connectivity between network devices, interfacesbetween devices are operational and properly connected, correctoperating systems are installed on devices, and power supplies areproperly connected. A method 800 for validating configuration of anetwork device is shown in FIG. 8. The validation process includes thebuildout service 122 issuing validation requests to the SNMP proxy 134Aand the hardware proxy 136A, as shown at block 802. Each validationrequest can specify a particular network device and requested data to bevalidated. In response to the validation requests, the SNMP proxy 134Aand hardware proxy 136A obtain data from the network devices usingaccess control data from the access control data store 128, as shown atblock 804 (e.g., using processes similar to those described above withreference to FIGS. 2 and 3). The SNMP proxy 134A and hardware proxy 136Areturn the requested validation data to the buildout service 122, asshown at block 806. In some instances, any restricted data is removedfrom the validation data before it is returned to the buildout service122. The buildout service 122 uses the received validation data tovalidate the proper configuration of the network devices, as shown atblock 808.

General Operating Environment

Having described various implements, an exemplary operating environmentsuitable for implementing aspects of the present disclosure is nowdescribed. Referring initially to FIG. 9 in particular, an exemplaryoperating environment for implementing aspects of the present disclosureis shown and designated generally as computing device 900. Computingdevice 900 is but one example of a suitable computing environment and isnot intended to suggest any limitation as to the scope of use orfunctionality of the technology described herein. Neither should thecomputing device 900 be interpreted as having any dependency orrequirement relating to any one or combination of componentsillustrated.

Aspects of the disclosure may be described in the general context ofcomputer code or machine-useable instructions, includingcomputer-executable instructions such as program modules, being executedby a computer or other machine, such as a personal data assistant orother handheld device. Generally, program modules including routines,programs, objects, components, data structures, etc., refer to code thatperform particular tasks or implement particular abstract data types.Aspects of the disclosure may be practiced in a variety of systemconfigurations, including hand-held devices, consumer electronics,general-purpose computers, more specialty computing devices, etc.Aspects of the disclosure may also be practiced in distributed computingenvironments where tasks are performed by remote-processing devices thatare linked through a communications network.

With reference to FIG. 9, computing device 900 includes a bus 910 thatdirectly or indirectly couples the following devices: memory 912, one ormore processors 914, one or more presentation components 916,input/output (I/O) ports 918, input/output components 920, and anillustrative power supply 922. Bus 910 represents what may be one ormore busses (such as an address bus, data bus, or combination thereof).Although the various blocks of FIG. 9 are shown with lines for the sakeof clarity, in reality, delineating various components is not so clear,and metaphorically, the lines would more accurately be grey and fuzzy.For example, one may consider a presentation component such as a displaydevice to be an I/O component. Also, processors have memory. Theinventors recognize that such is the nature of the art, and reiteratethat the diagram of FIG. 9 is merely illustrative of an exemplarycomputing device that can be used in connection with one or more aspectsof the present disclosure. Distinction is not made between suchcategories as “workstation,” “server,” “laptop,” “hand-held device,”etc., as all are contemplated within the scope of FIG. 9 and referenceto “computing device.”

Computing device 900 typically includes a variety of computer-readablemedia. Computer-readable media can be any available media that can beaccessed by computing device 900 and includes both volatile andnonvolatile media, removable and non-removable media. By way of example,and not limitation, computer-readable media may comprise computerstorage media and communication media. Computer storage media includesboth volatile and nonvolatile, removable and non-removable mediaimplemented in any method or technology for storage of information suchas computer-readable instructions, data structures, program modules orother data. Computer storage media includes, but is not limited to, RAM,ROM, EEPROM, flash memory or other memory technology, CD-ROM, digitalversatile disks (DVD) or other optical disk storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,or any other medium which can be used to store the desired informationand which can be accessed by computing device 900. Computer storagemedia does not comprise signals per se. Communication media typicallyembodies computer-readable instructions, data structures, programmodules or other data in a modulated data signal such as a carrier waveor other transport mechanism and includes any information deliverymedia. The term “modulated data signal” means a signal that has one ormore of its characteristics set or changed in such a manner as to encodeinformation in the signal. By way of example, and not limitation,communication media includes wired media such as a wired network ordirect-wired connection, and wireless media such as acoustic, RF,infrared and other wireless media. Combinations of any of the aboveshould also be included within the scope of computer-readable media.

Memory 912 includes computer-storage media in the form of volatileand/or nonvolatile memory. The memory may be removable, non-removable,or a combination thereof. Exemplary hardware devices include solid-statememory, hard drives, optical-disc drives, etc. Computing device 900includes one or more processors that read data from various entitiessuch as memory 912 or I/O components 920. Presentation component(s) 916present data indications to a user or other device. Exemplarypresentation components include a display device, speaker, printingcomponent, vibrating component, etc.

I/O ports 918 allow computing device 900 to be logically coupled toother devices including I/O components 920, some of which may be builtin. Illustrative components include a microphone, joystick, game pad,satellite dish, scanner, printer, wireless device, etc. The I/Ocomponents 920 may provide a natural user interface (NUI) that processesair gestures, voice, or other physiological inputs generated by a user.In some instance, inputs may be transmitted to an appropriate networkelement for further processing. A NUI may implement any combination ofspeech recognition, touch and stylus recognition, facial recognition,biometric recognition, gesture recognition both on screen and adjacentto the screen, air gestures, head and eye tracking, and touchrecognition associated with displays on the computing device 900. Thecomputing device 900 may be equipped with depth cameras, such as,stereoscopic camera systems, infrared camera systems, RGB camerasystems, and combinations of these for gesture detection andrecognition. Additionally, the computing device 900 may be equipped withaccelerometers or gyroscopes that enable detection of motion. The outputof the accelerometers or gyroscopes may be provided to the display ofthe computing device 900 to render immersive augmented reality orvirtual reality.

As can be understood, aspects of the technology described herein aregenerally directed to standardizing management across cloud computingenvironments subject to different data controls policies in a mannerthat maintains control over restricted data in the cloud computingenvironments. Aspects of the present disclosure have been described inrelation to particular configurations, which are intended in allrespects to be illustrative rather than restrictive. Alternativeconfigurations will become apparent to those of ordinary skill in theart to which the present disclosure pertains without departing from itsscope.

From the foregoing, it will be seen that the technology described hereinis one well adapted to attain all the ends and objects set forth above,together with other advantages which are obvious and inherent to thesystem and method. It will be understood that certain features andsubcombinations are of utility and may be employed without reference toother features and subcombinations. This is contemplated by and iswithin the scope of the claims.

What is claimed is:
 1. A computerized system comprising: a plurality ofcloud computing environments that are each subject to data controlpolicies to maintain control of restricted data in each cloud computingenvironment, each cloud computing environment including: a productionenvironment having a plurality of network devices, an access controldata store storing access control data required to access the networkdevices in the production environment, and one or more device accessservices that interact directly with the network devices to collect dataand/or issues commands using the access control data from the accesscontrol data store; and a remote cloud computing environment that isremote from each of the cloud computing environments and does not havepersistent access to restricted data in each cloud computingenvironment, the remote cloud computing environment including: one ormore execution services that implement workflows to manage aspects ofeach cloud computing environment by issuing requests to the one or moredevice access services to collect data from and/or issue commands to thenetwork devices in each cloud computing environment, and one or moresource of truth services that collect network configuration informationfor each cloud computing environment and make the network configurationinformation available to the one or more execution services.
 2. Thesystem of claim 1, wherein the one or more execution services include amonitoring service that employs at least one of the device accessservices to obtain telemetry data for the plurality of network devicesin each of the cloud computing environments.
 3. The system of claim 1,wherein the one or more execution services include a deployment servicethat employs at least one of the device access services to deploysoftware to one or more of the network devices in each of the cloudcomputing environments.
 4. The system of claim 1, wherein the one ormore execution services include a buildout service that employs at leastone of the device access services to configure one or more new networkdevices added to each of the cloud computing environments.
 5. The systemof claim 1, wherein the source of truth services include a network graphservice that builds a network graph from the network configurationinformation for each cloud computing environment.
 6. The system of claim1, wherein the source of truth services include a network state servicethat hosts information regarding a state of network devices in eachcloud computing environment.
 7. The system of claim 1, wherein eachcloud computing environment further comprises an access control servicethat controls external access to each cloud computing environment usingclaims-based access control.
 8. The system of claim 1, wherein thedevice access services include an SNMP proxy that obtains telemetry datafor network devices in each cloud computing environment using accesscontrol data to access the network devices.
 9. The system of claim 1,wherein the device access services include a hardware proxy that issuescommands to network devices in each cloud computing environment usingaccess control data to access the network devices.
 10. The system ofclaim 1, wherein the device access services include a UDP proxy thatcollects telemetry data pushed from network devices to the UDP proxy andpushes a portion of the telemetry data that does not include restricteddata to at least one of the one or more execution services.
 11. One ormore computer storage media storing computer-useable instructions that,when used by one or more computing devices, cause the one or morecomputing devices to perform operations comprising: receiving, from anexecution service in a first cloud computing environment at a deviceaccess service in a second cloud computing environment remote from thefirst cloud computing device, a request for an action to be performedfor a network device in the second cloud computing device, the executionservice not having access to restricted data in the second cloudcomputing environment; obtaining, by the device access service, accesscontrol data for the network device from an access control data store inthe second cloud computing environment; and issuing, by the deviceaccess service to the network device, one or more commands to performthe requested action on the network device using the access control datafor the network device.
 12. The one or more computer storage media ofclaim 11, wherein the request from the execution service received at thedevice access service is made via an API.
 13. The one or more computerstorage media of claim 11, wherein the request from the executionservice received at the device access service includes a claim, which isevaluated by an access control service in the second cloud computingenvironment to validate the claim.
 14. The one or more computer storagemedia of claim 11, wherein the request comprises a request for telemetrydata for the network device and the device access service comprises anSNMP proxy that obtains the telemetry data from the network device. 15.The one or more computer storage media of claim 11, wherein the deviceaccess service comprises a hardware proxy that performs an operation onthe network device in response to the request.
 16. A computerized systemcomprising: a first cloud computing environment subject to a datacontrol policy to maintain control of restricted data in the first cloudcomputing environment, the first cloud computing environment includingat least one device access service that interacts directly with networkdevices in the first cloud computing environment using access controldata maintained within the first cloud computing environment; and asecond cloud computing environment that does not have persistent accessto restricted data in the first cloud computing environment, the secondcloud computing environment including at least one execution servicethat does not have access to the access control data and that issuesrequests to the at least one device access service to cause the at leastone device access service to perform actions on the network devices inthe first cloud computing environment.
 17. The system of claim 16,wherein the at least one execution service includes one or more selectedfrom the following: a monitoring service that employs the at least onedevice access service to obtain telemetry data for the \network devicesin the first cloud computing environment; a deployment service thatemploys the at least one device access service to deploy software to thenetwork devices in the first cloud computing environment; and a buildoutservice that employs the at least one device access service to configureat least one new network device added to the first cloud computingenvironment.
 18. The system of claim 16, wherein the at least one deviceaccess service includes one or more selected from the following: an SNMPproxy that obtains telemetry data for the network devices in the firstcloud computing environment; a hardware proxy that performs operationson the network devices in the first cloud computing environment; and aUDP proxy that collects telemetry data pushed from the network devicesto the UDP proxy.
 19. The system of claim 16, wherein the second cloudcomputing environment further comprises at least one source of truthservice that collects network configuration information for the firstcloud computing environment and makes the network configurationinformation available to the at least one execution service.
 20. Thesystem of claim 19, wherein the at least one source of truth serviceincludes one or more selected from the following: a network graphservice that builds a network graph from the network configurationinformation for the first cloud computing environment; and a networkstate service that hosts information regarding a state of the networkdevices in the first cloud computing environment.